Federated Learning (FL) is a novel, multidisciplinary Machine Learning paradigm where multiple clients, such as mobile devices, collaborate to solve machine learning problems. Initially introduced in Kone{č}n{ý} et al. (2016a,b); McMahan et al. (2017), FL has gained further attention through its inclusion in the National AI Research and Development Strategic Plan (2023 Update) of the United States (Science and on Artificial Intelligence, 2023). The FL training process is inherently decentralized and often takes place in less controlled settings compared to data centers, posing unique challenges distinct from those in fully controlled environments. In this thesis, we identify five key challenges in Federated Learning and propose novel approaches to address them. These challenges arise from the heterogeneity of data and devices, communication issues, and privacy concerns for clients in FL training. Moreover, even well-established theoretical advances in FL require diverse forms of practical implementation to enhance their real-world applicability. Our contributions advance FL algorithms and systems, bridging theoretical advancements and practical implementations. More broadly, our work serves as a guide for researchers navigating the complexities of translating theoretical methods into efficient real-world implementations and software. Additionally, it offers insights into the reverse process of adapting practical implementation aspects back into theoretical algorithm design. This reverse process is particularly intriguing, as the practical perspective compels us to examine the underlying mechanics and flexibilities of algorithms more deeply, often uncovering new dimensions of the algorithms under study.

We present a response to the 2018 Request for Information (RFI) from the NITRD, NCO, NSF regarding the "Update to the 2016 National Artificial Intelligence Research and Development Strategic Plan." Through this document, we provide a response to the question of whether and how the National Artificial Intelligence Research and Development Strategic Plan (NAIRDSP) should be updated from the perspective of Fermilab, America's premier national laboratory for High Energy Physics (HEP). We believe the NAIRDSP should be extended in light of the rapid pace of development and innovation in the field of Artificial Intelligence (AI) since 2016, and present our recommendations below. AI has profoundly impacted many areas of human life, promising to dramatically reshape society --- e.g., economy, education, science --- in the coming years. We are still early in this process. It is critical to invest now in this technology to ensure it is safe and deployed ethically. Science and society both have a strong need for accuracy, efficiency, transparency, and accountability in algorithms, making investments in scientific AI particularly valuable. Thus far the US has been a leader in AI technologies, and we believe as a national Laboratory it is crucial to help maintain and extend this leadership. Moreover, investments in AI will be important for maintaining US leadership in the physical sciences.

As more decisions become automated and processed by algorithms, these processes become more opaque and less accountable. The public has a right to know the data processes that impact their lives so they can correct errors and contest decisions made by algorithms. Personal data collected from our social connections and online activities are used by the government and companies to make determinations about our ability to fly, obtain a job, get security clearance, and even determine the severity of criminal sentencing. These opaque, automated decision-making processes bear risks of secret profiling and discrimination as well as undermine our privacy and freedom of association. Without knowledge of the factors that provide the basis for decisions, it is impossible to know whether government and companies engage in practices that are deceptive, discriminatory, or unethical.

Artificial intelligence (AI) holds tremendous promise to benefit nearly all aspects of society, including the economy, healthcare, security, the law, transportation, even technology itself. On February 11, 2019, the President signed Executive Order 13859, Maintaining American Leadership in Artificial Intelligence. This order launched the American AI Initiative, a concerted effort to promote and protect AI technology and innovation in the United States. The Initiative implements a whole-of-government strategy in collaboration and engagement with the private sector, academia, the public, and like-minded international partners. Among other actions, key directives in the Initiative call for Federal agencies to prioritize AI research and development (R&D) investments, enhance access to high-quality cyberinfrastructure and data, ensure that the Nation leads in the development of technical standards for AI, and provide education and training opportunities to prepare the American workforce for the new era of AI.

Through this Request for Information (RFI), NITRD NCO seeks input from the public, including those directly performing Artificial Intelligence (AI) research and development (R&D) and directly affected by such R&D, on whether the strategic plan should be revised and, if so, the ways in which it may be improved. This includes suggestions as to the addition, removal, or modification of strategic aims, comments as to existing strategic aims as well as their past or future implementation by the Federal government. The public input provided in response to this RFI will inform NITRD NCO and the Select Committee in updating the National Artificial Intelligence Research and Development Strategic Plan. Interested persons are invited to submit comments on or before 11:59 p.m. (ET) on October 26, 2018. Instructions: Response to this RFI is voluntary.

Artificial intelligence (AI) is a transformative technology that holds promise for tremendous societal and economic benefit. AI has the potential to revolutionize how we live, work, learn, discover, and communicate. AI research can further our national priorities, including increased economic prosperity, improved educational opportunities and quality of life, and enhanced national and homeland security. Because of these potential benefits, the U.S. government has invested in AI research for many years. Yet, as with any significant technology in which the Federal government has interest, there are not only tremendous opportunities but also a number of considerations that must be taken into account in guiding the overall direction of Federally-funded R&D in AI.

In 2016, amidst this landscape of uncertainty, the United States government launched a series of activities and actions to help the country better understand and prepare for the impacts of advancements in artificial intelligence (Felten 2016b). In one of those actions, the government called for the creation of a national strategic plan on AI that defines the federal role in AI research and development (R&D). Why was a strategic plan needed? If done thoughtfully, a national AI R&D strategic plan could help address these uncertainties by identifying the federal role in AI investments and defining open AI R&D challenges that must be solved before AI can be used in important societal applications.

They also said that the government is in a better position than technology companies to invest in AI applications aimed at broad societal problems. They agreed that AI is in a crucial developmental moment, and that government has a unique opportunity to shape its future. That was the message from prominent AI technologists and researchers at a Senate committee hearing last week. The White House made a similar argument in a "National Artificial Intelligence Research and Development Strategic Plan," which it published in October. Senator Ted Cruz of Texas, who convened the hearing, expressed concern that the United States not cede its leadership in developing AI to China or any other foreign government.

Analysis Research in AI is expanding quickly, and the UK and US governments have begun to notice. Official reports about the new technology and future strategies were dropped by both governments this month. Blighty's Science and Technology Committee released Robotics and Artificial Intelligence, while the White House delivered Preparing for the Future of Artificial Intelligence and National Artificial Intelligence Research and Development Strategic Plan. The titles of the British and American reports provide a clue as to how both governments are responding. There is no "preparing" or "strategic plan" in the UK's reports.